Robust Multimodal Interaction in Challenging Environments
نویسندگان
چکیده
This paper briefly summarizes recent research documenting increased robustness that results from using multimodal interaction in environments that would challenge current graphical user interfaces, or even interfaces based on single natural communication modalities, such as speech or gesture. Multimodal interfaces that support the mutual disambiguation of modalities [1,2,3] can use one or more modalities or sources of information to overcome errors in another. The results presented in this paper demonstrate this phenomenon for circumstances in which users are highly exerted, and for 3D augmented and virtual reality environments that offer 4 sources of information.
منابع مشابه
Towards Robust Semantic Segmentation using Deep Fusion
Robust semantic scene understanding of unstructured environments is critical for robots operating in the real world. Several inherent natural factors such as shadows, glare and snow make this problem highly challenging, especially using RGB images. In this paper, we propose the use of multispectral and multimodal images to increase robustness of segmentation in real-world outdoor environments. ...
متن کاملRobust Semantic Segmentation using Deep Fusion
Robust semantic scene understanding of unstructured environments is critical for robots operating in the real world. Several inherent natural factors such as shadows, glare and snow make this problem highly challenging, especially using RGB images. In this paper, we propose the use of multispectral and multimodal images to increase robustness of segmentation in real-world outdoor environments. ...
متن کاملRobust Multimodal Command Interpretation for Human-Multirobot Interaction
In this work, we propose a multimodal interaction framework for robust human-multirobot communication in outdoor environments. In these scenarios, several human or environmental factors can cause errors, noise and wrong interpretations of the commands. The main goal of this work is to improve the robustness of human-robot interaction systems in similar situations. In particular, we propose a mu...
متن کاملDesigning robust multimodal systems for diverse users and environments
Multimodal interfaces are being developed that permit our highly skilled and coordinated communicative behavior to control system interactions in a more transparent and flexible interface experience than ever before. The presence of modality choice per se is an important feature and design issue for multimodal interfaces. As applications become more complex, a single modality does not permit va...
متن کاملA Robust and Generic Discourse Model for Multimodal Dialogue
We present an implemented, well-tested discourse model used in several monoas well as multimodal dialog systems. We provide a detailed description of the underlying representation and show how such challenging phenomena like cross-modal anaphoric expressions and ellipses are processed.
متن کامل